Patterns of Screening Recall Behavior Among Subspecialty Breast Radiologists.

RATIONALE AND OBJECTIVES: Determine whether there are patterns of lesion recall among breast imaging subspecialists interpreting screening mammography, and if so, whether recall patterns correlate to morphologies of screen-detected cancers. MATERIALS AND METHODS: This Institutional Review Board-approved, retrospective review included all screening examinations January 3, 2012-October 1, 2018 interpreted by fifteen breast imaging subspecialists at a large academic medical center and two outpatient imaging centers. Natural language processing identified radiologist recalls by lesion type (mass, calcifications, asymmetry, architectural distortion); proportions of callbacks by lesion types were calculated per radiologist. Hierarchical cluster analysis grouped radiologists based on recall patterns. Groups were compared to overall practice and each other by proportions of lesion types recalled, and overall and lesion-specific positive predictive value-1 (PPV1). RESULTS: Among 161,859 screening mammograms with 13,086 (8.1%) recalls, Hierarchical cluster analysis grouped 15 radiologists into five groups. There was substantial variation in proportions of lesions recalled: calcifications 13%-18% (Chi-square 45.69, p < 0.00001); mass 16%-44% (Chi-square 498.42, p < 0.00001); asymmetry 13%-47% (Chi-square 660.93, p < 0.00001) architectural distortion 6%-20% (Chi-square 283.81, p < 0.00001). Radiologist groups differed significantly in overall PPV1 (range 5.6%-8.8%; Chi-square 17.065, p = 0.0019). PPV1 by lesion type varied among groups: calcifications 9.2%-15.4% (Chi-square 2.56, p = 0.6339); mass 5.6%-8.5% (Chi-square 1.31, p = 0.8597); asymmetry 3.4%-5.9% (Chi-square 2.225, p = 0.6945); architectural distortion 5.6%-10.8% (Chi-square 5.810, p = 0.2138). Proportions of recalled lesions did not consistently correlate to proportions of screen-detected cancer. CONCLUSION: Breast imaging subspecialists have patterns for screening mammography recalls, suggesting differential weighting of imaging findings for perceived malignant potential. Radiologist recall patterns are not always predictive of screen-detected cancers nor lesion-specific PPV1s.

A Prospective Intervention to Reduce Burnout Among Academic Radiologists.

RATIONALE AND OBJECTIVES: Few studies have examined what constitutes effective interventions to reduce burnout among radiologists. We compared self-reported burnout among academic medical center radiologists before and after a series of departmental initiatives intended to increase wellbeing and professional satisfaction. MATERIALS AND METHODS: This Institutional Review Board-approved, prospective study took place 2017-2019 in a tertiary academic medical center. In pre- (2017) and post-intervention (2019) periods, we administered the previously-validated Stanford Physician Wellness Survey to faculty in our 11-division radiology department. Faculty rated their burnout level across 8 domains (professional fulfillment, emotional exhaustion, interpersonal disengagement, sleep difficulties, self-compassion, negative work impact on personal relations, organizational/personal values alignment, perceived quality of supervisory leadership). Between the two surveys, departmental initiatives focusing on culture, team building, work-life balance, and personal well-being were implemented (e.g., electronic medical record training, shorter work hours). Pre- and post-survey results were compared, using Whitney-Mann U test to calculate Z scores. RESULTS: Faculty members rated lower professional fulfillment (Z-3.04, p=0.002), higher emotional exhaustion (Z=2.52, p=0.012), increased sleep-related impairment (Z=2.38, p=0.012), and reduced organizational/personal values alignment (Z=-4.10, p<0.0001) between the two surveys. No significant differences were identified associated with interpersonal disengagement (Z=1.82, p=0.069), self-compassion (Z=1.39, p=0.164), negative impact of work on personal relationship (Z=0.89, p=0.372) and perceived supervisory leadership quality (Z=0.07, p=0.942). CONCLUSION: Despite numerous departmental initiatives intended to improve culture, workplace efficiency, work-life balance, and personal wellness, self-reported burnout was unchanged or worsened over time.Physician and employee wellness embedded into institutional culture maybe more effective than departmental improvement initiatives.

Self-reported Burnout: Comparison of Radiologists to Nonradiologist Peers at a Large Academic Medical Center.

RATIONALE AND OBJECTIVES: Relatively little data exist on factors associated with radiologists' burnout versus other medical specialties. We compared self-reported burnout among academic medical center radiologists versus nonradiologist peers to inform initiatives to increase wellbeing and professional satisfaction. MATERIALS AND METHODS: In 2017, our large urban academic medical center administered the Stanford Physician Wellness Survey to faculty in fifteen clinical departments (fourteen academic, one community-based). Faculty rated burnout via Likert scale (0-no burnout; 1-occasional stress/no burnout; 2-one or more burnout symptoms; 3-persistent burnout symptoms; 4-completely burned out); burnout defined as >=2. Responses in 11 domains (professional fulfillment, emotional exhaustion, interpersonal disengagement, sleep difficulties, self-compassion, negative work impact on personal relations, perceived appreciation, control over schedule, organizational/personal values alignment, electronic health record, perceived quality of supervisory leadership) compared radiologists versus nonradiologists for association with burnout, using Whitney-Mann U test to calculate Z scores. RESULTS: There was no significant difference in overall self-reported burnout between radiologists and nonradiologists, nor in self-rating for emotional exhaustion, interpersonal disengagement, self-compassion, control over schedule, organizational/personal values alignment, or electronic health record experience. Radiologists had significantly lower self-rating for work happiness (Z = -2.669, p = 0.0076), finding work meaningful (Z = -2.77351, p = 0.0055), perceiving physicians as highly valued (Z = -2.5486, p = 0.0108), and believing leadership treated them with respect and dignity (Z = -3.44149, p = 0.0006). CONCLUSION: Compared to nonradiologist colleagues, radiologists were less likely to find work meaningful and more likely to feel unhappy and undervalued in the workplace and by leadership. Initiatives to increase perceived appreciation, leadership relationships, and meaningfulness of work for radiologists may reduce burnout.

Variation in Radiologists' Follow-Up Imaging Recommendations for Small Cystic Pancreatic Lesions.

OBJECTIVE: This study aimed to determine the incidence, identify imaging and patient factors, and measure individual radiologist variation associated with follow-up recommendations for small focal cystic pancreatic lesions (FCPLs), a common incidental imaging finding. METHODS: This institutional review board-approved retrospective study analyzed 146,709 reports from abdominal CTs and MRIs performed in a large academic hospital from July 1, 2016, to June 30, 2018. A trained natural language processing tool identified 4,345 reports with FCPLs, which were manually reviewed to identify those containing one or more <1.5-cm pancreatic cysts. For these patients, patient, lesion, and radiologist features and follow-up recommendations for FCPL were extracted. A nonlinear random-effects model estimated degree of variation in follow-up recommendations across radiologists at department and division levels. RESULTS: Of 2,872 reports with FCPLs < 1.5 cm, 708 (24.7%) had FCPL-related follow-up recommendations. Average patient age was 67 years (SD ± 11). In all, 1,721 (60.0%) reports were for female patients; 59.3% of patients had only one cyst. In multivariable analysis, older patients had slightly lower follow-up recommendation rates (odds ratio [OR]: 0.98 [0.98-1.00] per additional year), and lesions associated with main duct dilatation and septation were more likely to have a follow-up recommendation (ORs: 1.93 [1.11-3.36] and 2.88 [1.89-4.38], respectively). Radiologist years in practice (P = .51), trainee presence (P = .21), and radiologist gender (P = .52) were not associated with increased follow-up recommendations. There was significant interradiologist variation in the Abdominal Imaging Division (P = .04), but not in Emergency Radiology (P = .31) or Cancer Imaging Divisions (P = .29). DISCUSSION: Interradiologist variation significantly contributes to variability in follow-up imaging recommendations for FCPLs.

Radiologist Variation in the Rates of Follow-up Imaging Recommendations Made for Pulmonary Nodules.

OBJECTIVE: Determine whether differences exist in rates of follow-up recommendations made for pulmonary nodules after accounting for multiple patient and radiologist factors. METHODS: This Institutional Review Board-approved, retrospective study was performed at an urban academic quaternary care hospital. We analyzed 142,001 chest and abdominal CT reports from January 1, 2016, to December 31, 2018, from abdominal, thoracic, and emergency radiology subspecialty divisions. A previously validated natural language processing (NLP) tool identified 24,512 reports documenting pulmonary nodule(s), excluding reports NLP-positive for lung cancer. A second validated NLP tool identified reports with follow-up recommendations specifically for pulmonary nodules. Multivariable logistic regression was used to determine the likelihood of pulmonary nodule follow-up recommendation. Interradiologist variability was quantified within subspecialty divisions. RESULTS: NLP classified 4,939 of 24,512 (20.1%) reports as having a follow-up recommendation for pulmonary nodule. Male patients comprised 45.3% (11,097) of the patient cohort; average patient age was 61.4 years (±14.1 years). The majority of reports were from outpatient studies (62.7%, 15,376 of 24,512), were chest CTs (75.9%, 18,615 of 24,512), and were interpreted by thoracic radiologists (63.7%, 15,614 of 24,512). In multivariable analysis, studies for male patients (odds ratio [OR]: 0.9 [0.8-0.9]) and abdominal CTs (OR: 0.6 [0.6-0.7] compared with chest CT) were less likely to have a pulmonary nodule follow-up recommendation. Older patients had higher rates of follow-up recommendation (OR: 1.01 for each additional year). Division-level analysis showed up to 4.3-fold difference between radiologists in the probability of making a follow-up recommendation for a pulmonary nodule. DISCUSSION: Significant differences exist in the probability of making a follow-up recommendation for pulmonary nodules among radiologists within the same subspecialty division.

Appropriateness of inpatient stress testing: Implications for development of clinical decision support mechanisms and future criteria.

BACKGROUND: An upcoming national mandate will require consultation of appropriate use criteria (AUC) through a clinical decision support mechanism (CDSM) for advanced imaging. We aimed to evaluate our current ability to ascertain test appropriateness. METHODS: We prospectively collected data on 288 consecutive stress tests and coronary computed tomography angiography studies for medical inpatients. Study appropriateness was determined independently by two physicians using the 2013 Multimodality AUC. RESULTS: The median age of the study population was 66 years [interquartile range (IQR) 56, 75], 40.8% were female, and 52.8% had a history of coronary artery disease. Review of the electronic health record (EHR) alone was sufficient to deem appropriateness for 87.2% of cases. The most common reason it was insufficient was inability to determine if the patient could exercise (59.5%). After reviewing the EHR and pilot CDSM data together, appropriateness could be determined for 95.8% of the cases. The most common reason appropriateness could not be determined was that the exam indication was not addressed by an AUC criterion (83.3%). CONCLUSION: In preparing for the mandate, it will be important for future CDSM to obtain information on the patient's ability to exercise and for future AUC to include additional indications that are not currently addressed.

Predictors of Self-Reported Burnout Among Radiology Faculty at a Large Academic Medical Center.

OBJECTIVE: Determine predictors of self-reported burnout among academic radiologists. METHODS: In 2017, radiologists at an urban medical center completed the Stanford Wellness Survey, rating burnout via Likert scale (0: no burnout; 1: occasional stress, no burnout; 2: one or more burnout symptoms; 3: persistent burnout symptoms; 4: completely burned out). Univariate analyses assessed age, gender, family situation, clinical versus research focus, and academic rank for association with burnout (Likert scale ≥ 2). Responses in 11 domains querying definitions of burnout (professional fulfillment, emotional exhaustion, interpersonal disengagement), individual factors (sleep-related impairment, self-compassion, negative work impact on personal relationships), institutional factors (perceived appreciation, control over schedule, organizational or personal values alignment, electronic health record experience, supervisor's leadership quality) were evaluated for association with burnout, using χ2 and logistic regression to calculate odds ratios (ORs). RESULTS: In 159 of 204 (77.9%) completed radiologist surveys, 35.2% (56 of 159) reported burnout. Age < 40 years (P = .0068) and clinical focus (P = .0111) were significantly associated with burnout. In univariate analysis, all domains except electronic health record were statistically significant: emotional exhaustion (OR = 1.93, P < .0001); professional fulfillment (OR = 0.78, P < .0001); self-compassion (OR = 1.36, P < .0001); perceived appreciation (OR = 0.78, P < .0001); sleep-related impairment (OR = 1.20, P < .0001); supervisor's leadership quality (OR = 0.91, P < .0001); interpersonal disengagement (OR = 1.31, P < .0001); organizational or personal values alignment (OR = 0.87, P = .0004); negative work impact on personal relationships (OR = 1.10, P = .0070); control over schedule (OR = 0.80, P = .0054); electronic health record experience (OR=1.03, P = .5392). DISCUSSION: Nearly all questions significantly predicted self-reported burnout, observed in over one-third of academic radiologists. Younger age and clinical focus were associated with burnout. Initiatives targeting individual factors (eg, sleep impairment, self-compassion) and institutional factors (eg, physician appreciation, leadership-faculty interactions) may reduce burnout.

Can emergency department provider notes help to achieve more dynamic clinical decision support?

OBJECTIVE: Assess whether clinical data were present in emergency department (ED) provider notes at time of order entry for cervical spine (c-spine) imaging that could be used to augment or pre-populate clinical decision support (CDS) attributes. METHODS: This Institutional Review Board-approved retrospective study, performed in a quaternary hospital, included all encounters for adult ED patients seen April 1, 2013-September 30, 2014 for a chief complaint of trauma who received c-spine computed tomography (CT) or x-ray. We assessed proportion of ED encounters with at least 1 c-spine-specific CDS rule attribute in clinical notes available at the time of imaging order and agreement between attributes in clinical notes and data entered into CDS. RESULTS: A portion of the clinical note was submitted before imaging order in 42% (184/438) of encounters reviewed; 59.2% (109/184) of encounters with note portions submitted before imaging order had at least 1 positive CDS attribute identified supporting imaging study appropriateness; 34.8% (64/184) identified exclusion criteria where CDS appropriateness recommendations would not be applicable. 65.8% (121/184) of encounters had either a positive CDS attribute or an exclusion criterion. Concordance of c-spine CDS attributes when present in both notes and CDS was 68.4% (κ = 0.35 95% CI: 0.15-0.56; McNemar P = 0.23). CONCLUSIONS: Clinical notes are an underutilized source of clinical attributes needed for CDS, available in a substantial percentage of encounters at the time of imaging order. Automated pre-population of imaging order requisitions with relevant clinical information extracted from electronic health record provider notes may: (1) improve ordering efficiency by reducing redundant data entry, (2) help improve clinical relevance of CDS alerts, and (3) potentially reduce provider burnout from extraneous alerts.

Comparing Tumor Characteristics and Rates of Breast Cancers Detected by Screening Digital Breast Tomosynthesis and Full-Field Digital Mammography.

OBJECTIVE. The purpose of this study was to compare the cancer detection rates (CDRs), tumor types, and characteristics between screening digital breast tomosynthesis (DBT) and screening full-field digital mammography (FFDM) in a matched patient population in a large academic breast imaging practice with mixed DBT and FFDM technology. MATERIALS AND METHODS. In this retrospective study, we reviewed consecutive screening FFDM and DBT examinations performed between October 2012 and September 2014. To control for nonrandomized selection of FFDM versus DBT examinations, we applied propensity score matching on the basis of patient age, imaging site, and prior imaging findings. An institutional breast cancer registry identified cancer diagnoses. CDR and tumor type, grade, receptor, nodal status, and size were compared between matched FFDM and DBT groups. RESULTS. Sixty-one cancers were detected in the matched screening cohort of DBT (n = 9817) and FFDM (n = 14,180) examinations. CDR was higher with DBT than with FFDM for invasive cancers (2.8 vs 1.3, p = 0.01), minimal cancers (2.4 vs 1.2, p = 0.03), estrogen receptor-positive invasive cancers (2.6 vs 1.1, p = 0.01), and node-negative invasive cancers (2.3 vs 1.1, p = 0.02.), respectively. The ratio of screen-detected invasive cancers to ductal carcinoma in situ on DBT (3.0) was not significantly different from that on FFDM (2.6) (p = 0.79). CONCLUSION. DBT results in an overall increase in CDR irrespective of the tumor type, size, or grade of cancer.

Fast Healthcare Interoperability Resources, Clinical Quality Language, and Systematized Nomenclature of Medicine-Clinical Terms in Representing Clinical Evidence Logic Statements for the Use of Imaging Procedures: Descriptive Study.

BACKGROUND: Evidence-based guidelines and recommendations can be transformed into "If-Then" Clinical Evidence Logic Statements (CELS). Imaging-related CELS were represented in standardized formats in the Harvard Medical School Library of Evidence (HLE). OBJECTIVE: We aimed to (1) describe the representation of CELS using established Systematized Nomenclature of Medicine-Clinical Terms (SNOMED CT), Clinical Quality Language (CQL), and Fast Healthcare Interoperability Resources (FHIR) standards and (2) assess the limitations of using these standards to represent imaging-related CELS. METHODS: This study was exempt from review by the Institutional Review Board as it involved no human subjects. Imaging-related clinical recommendations were extracted from evidence sources and translated into CELS. The clinical terminologies of CELS were represented using SNOMED CT and the condition-action logic was represented in CQL and FHIR. Numbers of fully and partially represented CELS were tallied. RESULTS: A total of 765 CELS were represented in the HLE as of December 2018. We were able to fully represent 137 of 765 (17.9%) CELS using SNOMED CT, CQL, and FHIR. We were able to represent terms using SNOMED CT in the temporal component for action ("Then") statements in CQL and FHIR in 755 of 765 (98.7%) CELS. CONCLUSIONS: CELS were represented as shareable clinical decision support (CDS) knowledge artifacts using existing standards-SNOMED CT, FHIR, and CQL-to promote and accelerate adoption of evidence-based practice. Limitations to standardization persist, which could be minimized with an add-on set of standard terms and value sets and by adding time frames to the CQL framework.

Variation in Follow-up Imaging Recommendations in Radiology Reports: Patient, Modality, and Radiologist Predictors.

Background Variation between radiologists when making recommendations for additional imaging and associated factors are, to the knowledge of the authors, unknown. Clear identification of factors that account for variation in follow-up recommendations might prevent unnecessary tests for incidental or ambiguous image findings. Purpose To determine incidence and identify factors associated with follow-up recommendations in radiology reports from multiple modalities, patient care settings, and imaging divisions. Materials and Methods This retrospective study analyzed 318 366 reports obtained from diagnostic imaging examinations performed at a large urban quaternary care hospital from January 1 to December 31, 2016, excluding breast and US reports. A subset of 1000 reports were randomly selected and manually annotated to train and validate a machine learning algorithm to predict whether a report included a follow-up imaging recommendation (training-and-validation set consisted of 850 reports and test set of 150 reports). The trained algorithm was used to classify 318 366 reports. Multivariable logistic regression was used to determine the likelihood of follow-up recommendation. Additional analysis by imaging subspecialty division was performed, and intradivision and interradiologist variability was quantified. Results The machine learning algorithm classified 38 745 of 318 366 (12.2%) reports as containing follow-up recommendations. Average patient age was 59 years ± 17 (standard deviation); 45.2% (143 767 of 318 366) of reports were from male patients. Among 65 radiologists, 57% (37 of 65) were men. At multivariable analysis, older patients had higher rates of follow-up recommendations (odds ratio [OR], 1.01 [95% confidence interval {CI}: 1.01, 1.01] for each additional year), male patients had lower rates of follow-up recommendations (OR, 0.9; 95% CI: 0.9, 1.0), and follow-up recommendations were most common among CT studies (OR, 4.2 [95% CI: 4.0, 4.4] compared with radiography). Radiologist sex (P = .54), presence of a trainee (P = .45), and years in practice (P = .49) were not significant predictors overall. A division-level analysis showed 2.8-fold to 6.7-fold interradiologist variation. Conclusion Substantial interradiologist variation exists in the probability of recommending a follow-up examination in a radiology report, after adjusting for patient, examination, and radiologist factors. © RSNA, 2019 See also the editorial by Russell in this issue.

Use of Imaging in the Emergency Department: Do Individual Physicians Contribute to Variation?

OBJECTIVE. The purpose of this study is to investigate the magnitude of physician variation in the use of imaging and the factors associated with variation in an urban emergency department (ED) in the United States. MATERIALS AND METHODS. This retrospective cohort study was conducted from April 1, 2013, to March 31, 2014, in the ED of a level I adult trauma center in the northeastern United States. The study cohort included all patient visits to the ED during the study period. We built hierarchic and logistic regression models to determine per-physician utilization of low- and high-cost imaging, and we identified factors correlated with variation in use. Global (i.e., intraclass correlation coefficient) and individual variability metrics were used to profile physician variation after controlling for patient-, visit-, and physician-related covariates. RESULTS. A total of 56,793 patients presented to the ED during the study; of these patients, 49.5% (28,135) underwent imaging, with 38.2% (21,686) undergoing low-cost imaging and 21.9% (12,430) undergoing high-cost imaging. Statistically significant predictors of imaging orders were patient age and sex, number of secondary diagnoses, certain primary diagnoses, time of arrival in the ED, and ED crowding. Unadjusted and adjusted intraclass correlation coefficients were 0.0072 and 0.0066, respectively, for low-cost imaging, and 0.0097 and 0.0090, respectively, for high-cost imaging. The coefficient of variation for adjusted imaging odds ratios was 10.9% and 14.0% for low- and high-cost imaging, respectively, indicating a moderate degree of variation. CONCLUSION. Unexplained and moderate variation in imaging utilization exists among ED physicians, even after controlling for patient, visit, and physician characteristics. Improvement initiatives using well-defined ED imaging quality measures may help improve quality and reduce waste.

An Electronic Health Record Order Entry-Enabled Educational Intervention Is Not Effective in Reducing STAT Inpatient Radiology Orders.

OBJECTIVE: Assess whether introducing order priorities with defined performance expectations in the electronic health record (EHR) reduces immediate inpatient radiology orders. MATERIALS AND METHODS: This Institutional Review Board-approved, retrospective study was performed at a 776-bed academic hospital conducting 164,000+ inpatient radiology examinations annually. Study period was January 2, 2017, to July 23, 2017; 14 weeks pre- and postimplementation of an education-only intervention including replacing urgent and as soon as possible priorities with imaging within next 6, 12, or 24 hours; imaging in the morning; and required for discharge priorities. STAT routine, timed, today order priorities remained unchanged. Institution-wide training immediately pre- and postimplementation was provided through two waves of e-mail and electronic tip sheets. Primary outcome measure was total STAT studies ordered of total radiology studies ordered per week (STAT rate). Secondary outcomes were non-STAT, non-routine (non-SR) order rate, and routine order rate. Paired t test and statistical process control (SPC) analysis were performed. RESULTS: STAT rate pre- (22.5%, 7,150 STAT of 31,765 total; weeks 1-14) and postintervention (23.4%, 7,481 STAT of 32,034 total; weeks 16-29) remained unchanged (P = .37). SPC demonstrated no special cause variation. Postintervention non-SR rate increased 3-fold (2.7%, 859 non-SR of 31,765 total pre-intervention versus 8.2%, 2,615 non-SR of 32,034 total postintervention; 8.2%/2.7% = 3.0; P < .0001). There was an 8.8% relative reduction in routine rate postintervention (73.9%, 23,471 routine of 31,765 total pre-intervention; 67.4%, 21,579 routine of 32,034 total postintervention; (73.9% - 67.4%)/73.9% × 100 = 8.8%; P < .0001). CONCLUSION: Implementing ordering priorities with defined performance expectations in the EHR reduced routine but did not reduce STAT inpatient radiology orders. More stringent interventions may be needed to reduce unnecessary STAT inpatient radiology ordering to improve use of limited imaging resources.

Can Automated Retrieval of Data from Emergency Department Physician Notes Enhance the Imaging Order Entry Process?

BACKGROUND: When a paucity of clinical information is communicated from ordering physicians to radiologists at the time of radiology order entry, suboptimal imaging interpretations and patient care may result. OBJECTIVES: Compare documentation of relevant clinical information in electronic health record (EHR) provider note to computed tomography (CT) order requisition, prior to ordering of head CT for emergency department (ED) patients presenting with headache. METHODS: In this institutional review board-approved retrospective observational study performed between April 1, 2013 and September 30, 2014 at an adult quaternary academic hospital, we reviewed data from 666 consecutive ED encounters for patients with headaches who received head CT. The primary outcome was the number of concept unique identifiers (CUIs) relating to headache extracted via ontology-based natural language processing from the history of present illness (HPI) section in ED notes compared with the number of concepts obtained from the imaging order requisition. RESULTS: Our analysis was conducted on cases where the HPI note section was completed prior to image order entry, which occurred in 23.1% (154/666) of encounters. For these 154 encounters, the number of CUIs specific to headache per note extracted from the HPI (median = 3, interquartile range [IQR]: 2-4) was significantly greater than the number of CUIs per encounter obtained from the imaging order requisition (median = 1, IQR: 1-2; Wilcoxon signed rank p < 0.0001). Extracted concepts from notes were distinct from order requisition indications in 92.9% (143/154) of cases. CONCLUSION: EHR provider notes are a valuable source of relevant clinical information at the time of imaging test ordering. Automated extraction of clinical information from notes to prepopulate imaging order requisitions may improve communication between ordering physicians and radiologists, enhance efficiency of ordering process by reducing redundant data entry, and may help improve clinical relevance of clinical decision support at the time of order entry, potentially reducing provider burnout from extraneous alerts.

Impact of a Multifaceted Information Technology-Enabled Intervention on the Adoption of ACR White Paper Follow-Up Recommendations for Incidental Adnexal Lesions Detected on CT.

OBJECTIVE. The objective of our study was to improve adherence to American College of Radiology (ACR) white paper follow-up imaging recommendations for incidental adnexal lesions seen on pelvic CT (herein referred to as "adherence rate to recommendations"). MATERIALS AND METHODS. This quality improvement project was conducted at a large academic teaching hospital. The baseline adherence rate to recommendations was assessed by screening all pelvic CT reports for the period from October 22, 2016, through December 22, 2016, for incidental adnexal findings, followed by manual review. Forty abdominal and cancer imaging radiologists were surveyed to understand the barriers to adoption of the recommendations. Interventions to address the most common identified barriers were implemented on December 23, 2016. The postintervention adherence rate was assessed from December 23, 2016, through February 15, 2017, by again screening CT pelvis reports for incidental adnexal lesions followed by manual review. The change in pre- and postintervention adherence rates was assessed using the Fisher exact test and statistical process control (SPC) p-chart with 3-sigma control limits. RESULTS. The adherence rate to recommendations at baseline was 67% (121/181). Of the 28 of 40 (70%) radiologists who completed the survey, only 29% (8/28) often or consistently used the recommendations. Not remembering the details of the recommendations or not having time to look them up accounted for 83.3% of the barriers cited by radiologists. Interventions consisted of radiologist education and creation of an easily accessible clinical decision support tool incorporated into radiology reporting workflow. The adherence rate to recommendations after the intervention increased to 87% (129/148; p < 0.0001), as also shown by the SPC chart. CONCLUSION. The rate of adherence to follow-up imaging recommendations significantly increased after radiologist education and incorporation of recommendations into the radiologist workflow.

Patient, Radiologist, and Examination Characteristics Affecting Screening Mammography Recall Rates in a Large Academic Practice.

OBJECTIVE: The aims of this study were to evaluate patient, radiologist, and examination characteristics affecting screening mammography recall rates in an academic breast imaging practice and to identify modifiable factors that could reduce recall variation. METHODS: This institutional review board-approved retrospective study included screening mammographic examinations in female patients interpreted by 13 breast imaging specialists at an academic center and two outpatient centers from October 1, 2012, to May 31, 2015. Patient demographics were extracted via electronic medical record. Natural language processing captured breast density, BI-RADS assignment, and current and prior screening examination findings. Radiologists' annual screening volumes, clinical experience, and concentration in breast imaging were calculated. Risk aversion, stress from uncertainty, and malpractice concerns were derived via survey. Univariate and multivariate analyses assessed patient, radiologist, and examination characteristics associated with likelihood of mammography recall. The Pearson product-moment correlation coefficient was used to assess the relationship between cancer detection rate and recall rate. RESULTS: Overall, 5,678 of 61,198 screening examinations (9.3%) were recalled. In multivariate analysis, patient and radiologist characteristics associated with higher odds of recall included patient's age < 50 years (P < .0001), prior mammographic findings (calcification [P < .0001], mass [P < .0001], higher density category [P < .0001]), baseline examination (P < .0001), annual reading volume < 1,250 examinations (P = .0282), and <10 years of experience (P = .0036). Radiologist's risk aversion, stress from uncertainty, malpractice concerns, and cancer detection rates were not associated with higher recall rates (r = -0.36, P = .23). CONCLUSIONS: In addition to patient and examination factors, screening recall variations were associated with radiologists' annual reading volume and experience. Interventions targeting radiologist factors (screening volumes, second review of potential recalls) may help reduce unwarranted variation in screening recall.

Effect of Clinical Decision Support-Generated Report Cards Versus Real-Time Alerts on Primary Care Provider Guideline Adherence for Low Back Pain Outpatient Lumbar Spine MRI Orders.

OBJECTIVE: The purpose of this study is to determine whether the type of feedback on evidence-based guideline adherence influences adult primary care provider (PCP) lumbar spine (LS) MRI orders for low back pain (LBP). MATERIALS AND METHODS: Four types of guideline adherence feedback were tested on eight tertiary health care system outpatient PCP practices: no feedback during baseline (March 1, 2012-October 4, 2012), randomization by practice to either clinical decision support (CDS)-generated report cards comparing providers to peers only or real-time CDS alerts at order entry during intervention 1 (February 6, 2013-December 31, 2013), and both feedback types for all practices during intervention 2 (January 14, 2014-June 20, 2014, and September 4, 2014-January 21, 2015). International Classification of Disease codes identified LBP visits (excluding Medicare fee-for-service). The primary outcome of the likelihood of LS MRI order being made on the day of or 1-30 days after the outpatient LBP visit was adjusted by feedback type (none, report cards only, real-time alerts only, or both); patient age, sex, race, and insurance status; and provider sex and experience. RESULTS: Half of PCPs (54/108) remained for all three periods, conducting 9394 of 107,938 (8.7%) outpatient LBP visits. The proportion of LBP visits increased over the course of the study (p = 0.0001). In multilevel hierarchic regression, report cards resulted in a lower likelihood of LS MRI orders made the day of and 1-30 days after the visit versus baseline: 38% (p = 0.009) and 37% (p = 0.006) for report cards alone, and 27% (p = 0.020) and 27% (p = 0.016) with alerts, respectively. Real-time alerts alone did not affect MRI orders made the day of (p = 0.585) or 1-30 days after (p = 0.650) the visit. No patient or provider variables were associated with LS MRI orders being generated on the day of or 1-30 days after the LBP visit. CONCLUSION: CDS-generated evidence-based report cards can substantially reduce outpatient PCP LS MRI orders on the day of and 1-30 days after the LBP visit. Real-time CDS alerts do not.

Classifying Safety Events Related to Diagnostic Imaging From a Safety Reporting System Using a Human Factors Framework.

PURPOSE: The aim of this study was to measure diagnostic imaging safety events reported to an electronic safety reporting system and assess steps at which they occurred within the diagnostic imaging workflow and contributing sociotechnical factors. METHODS: The authors evaluated all electronic safety reporting system reports related to diagnostic imaging during calendar year 2015 at an academic medical center with 50,000 admissions, 950,000 ambulatory visits, and 680,000 diagnostic imaging studies annually. Each report was assigned a harm score ranging from 0 to 4 by the reporter; scores of 2 (minor harm) to 4 (death) were classified as "potential harm." Two reviewers manually classified reports into steps involved in the diagnostic imaging chain and sociotechnical factors per the Systems Engineering Initiative for Patient Safety framework. The κ coefficient was used to measure interreviewer agreement on 10% of reports. The percentage of reports that could cause "potential harm" was compared for each step and sociotechnical factor using χ2 analysis. RESULTS: Of 11,570 safety reports submitted in 2015, 854 (7%) were related to diagnostic imaging. Although the most common step was imaging procedure (54% of reports), potential harm occurred more in result communication (odds ratio, 2.36; P = .05). Person factors most commonly contributed to safety reports (71%). Potential harm occurred more in safety reports that were related to tasks compared with person factors (odds ratio, 5.03; P < .0001). The κ coefficient was 0.79. CONCLUSIONS: Safety events were related to diagnostic imaging in 7% of reported events. Potential harm occurred primarily during imaging procedure and result communication. Safety events were attributed to multifactorial sociotechnical factors. Further work is necessary to decrease safety events related to diagnostic imaging.